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ABSTRACT 



This essay derives from roundtable discussions of teams from 
seven research universities involved in a Knight Collaborative Engagement on 
Academic Indicators and Information Systems to Guide Resource Decisions. 

While colleges and universities are built on the premise that good data, well 
tested by an often adversarial process of discovery and verification, can 
yield fundamental truths about how things work and how they ought to, most 
institutions have yet to learn how to use data strategically. The practice 
uses data for narrow and parochial causes, often to protect the existing 
order; seldom is data used as the instrument of strategy, that is, to gauge 
the capacity of an institution to fulfill current commitments or pursue new 
opportunities, to understand external markets and the competition for new or 
existing programs and services, to analyze the opportunities for new ventures 
through collaboration among departments and centers, or to explore the 
prospect of even broader collaboration with other colleges and universities. 
The essay suggests use of a matrix of domains and capacities to support 
strategic decision making and development of a national data base centered on 
collective performance to demonstrate responsibility for service to the 
public. (RH) 
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I 



t is a story often told. 
The chair convenes 
the committee, turns to 
his colleague who has 
developed a proposal, 
and asks for a succinct 
summary of the issue. 
Next comes a discus- 
sion of the proposal’s 
pros and cons, the costs 
associated with acting 
now or letting time fix 
the problem. Just as the 
discussion gathers mo- 
mentum, there comes 
from the end of the 
table the gentle but 
authoritative “Ahem,” and the ever-so-polite query: 
“Tell me, George, have we got any data to support 
that argument?” 

It is a time-tested stratagem, this move to dis- 
miss — not by openly opposing an idea but by insisting 
that nothing be ventured until sufficient data are at 
hand to evaluate every consequence, both intended 
and unintended, until the solution can be ascertained as 
the very best, not just for the moment, but for all time. 
What the chair, every committee member, and espe- 
cially the proposer know is that this idea is going 
nowhere — not now, not this month, probably not this 
year. 

The irony is that colleges and universities are just 
as likely to make a decision on the basis of little or no 
data. It is not unusual for a proposal that offers 
breath-taking opportunity to come before the leadership 
group: perhaps a major donor has appeared on the 
horizon, offering start-up funding to build a facility, 



develop a new graduate program, or expand the insti- 
tution’s presence in the distance-learning market. The 
advantages to proceeding appear self-evident at 
worst, overwhelming at best. The discussion is mini- 
mal, upbeat, and to the point: the proposal is 
approved, the decision made, the major venture 
begun with scarcely a thought about the market for 
the new initiative, the long-term impact in terms of 
human or financial resources, or the effect on institu- 
tional focus and the quality of existing programs. 

T hese stories convey much about the irony and 
ambivalence that swirl around the uses of data in 
an academic setting. Colleges and universities, as 
scholarly communities, are built on the premise that 
numbers matter, that good data — those that are well 
tested by an often adversarial process of discovery 
and verification — can yield fundamental truths about 
how things work and, by extension, how things ought 
to be. But if the appetite for information is insatiable, 
most institutions have yet to learn how to use data 
strategically. The pervasive practice is all too often to 
enlist data in narrow and parochial causes — to fight turf 
battles, impede change, secure the dominance of 
ideas or personalities within a domain, or justify past 
and current actions at the unit or institutional level. It 
is a disposition concerned mainly with protecting the 
existing order by identifying and averting potential 
problems. 

All too seldom do institutions make data the 
instruments of strategy in the fullest sense — to gauge 
the capacity of an institution to fulfill current commit- 
ments or pursue new opportunities, to understand 
external markets and the competition for new or 
existing programs and services, to analyze the oppor- 
tunities for new ventures through collaboration 
among departments and centers, or to explore the 
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prospect of even broader collaboration with other col- 
leges and universities. While higher education 
exhibits a preoccupation with numbers like never 
before, institutions find themselves oddly stretched to 
both extremes of a spectrum expressed by these two 
apocryphal tales. On the one hand, “We deferred any 
decision for want of sufficient data.” And on the other, 
“We made the decision, data be damned.” The ironic 



Still lacking is the ability to draw data 
effectively into a process of responsible 
judgment and decision making within an 
institution — to make thoughtful use of data 
as a gauge of capacity and prospects. 



result is an institution that uses data extraordinarily 
well when rendering scholarly judgment but too often 
fails to use data effectively to improve its own opera- 
tions, processes, and prospects. 

T his Policy Perspectives fundamentally asks: What 
would have to change to make the use of data in the 
management of colleges and universities more strate- 
gic and less defensive? How might colleges and uni- 
versities foster a culture of data that supports the busi- 
ness of making decisions in an academic enterprise? 
What opportunities exist for using data in ways that 
benefit higher education institutions collectively as 
well as individually? Our answers grow out of a 
roundtable composed principally of teams from seven 
research universities that had worked together in a 
Knight Collaborative Engagement on Academic Indi- 
cators and Information Systems to Guide Resource 
Decisions. What we came to understand is that most 
major research universities have gotten substantially 
better at assembling good transactional data for moni- 
toring basic operations. Still lacking is the ability to 
draw data effectively into a process of responsible 
judgment and decision making within an institution — 
to make thoughtful use of data as a gauge of capacity 
and prospects both within the institution and among 
higher education institutions as a whole. 



Deciphering 

We begin with a simple observation. Colleges 
and universities today are awash with data — reports, 
online databases, data warehouses, sophisticated post- 



ing and inquiry systems — all attesting to higher edu- 
cation’s embrace of information technology. And still 
the problem, as most faculty and nearly all adminis- 
trators have come to understand, is that data too seldom 
come to engage with larger conceptions of the institu- 
tion’s direction — its capacity, its challenges and 
opportunities, and its decision-making process. Univer- 
sities and colleges often expend energy collecting 
data for narrow and even myopic purposes. Many of the 
data compiled in such instances are incomplete, erro- 
neous, based on questionable assumptions — more a 
hindrance than a help to institutional strategy. 

C olleges and universities assemble data for a vari- 
ety of purposes. There are public relations data, 
honed and displayed in contexts carefully designed to 
present an institution in its most flattering light. There 
are accountability data, assembled at the mandate of a 
governing board or legislature eager to measure the 
value an institution contributes to the lives of its grad- 
uates and to society. And then there are the data an 
institution generates in the process of monitoring its 
own operations. 

Those who collect such data are ordinarily their 
owners — the department or unit that defines the data 
elements, monitors their collection and storage, and 
decides to whom and for what purposes the data are 
made available. The result is a collected database 
whose elements are guarded, isolated, and frequently 
inconsistent. Data definitions become turf definitions, 
and building crosswalks between one data source and 



Data definitions become turf defini- 
tions, and building crosswalks between 
one data source and another remains 
an uncertain endeavor. 



another remains an uncertain endeavor. Who’s asking 
becomes as important as who’s answering. Data that the 
government relations people prepare in response to 
questions posed by a state legislature might well differ 
from internal management numbers. Even a seem- 
ingly straightforward question, such as how many 
full-time faculty the institution employs, can yield 
embarrassingly different answers. 

Data assembled around such differing centers of 
gravity seldom enhance an institution’s ability to 
make informed judgments about its direction and 
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priorities. Given the dispersion of authority and 
responsibility, it is not surprising that data in transit 
from one administrative domain to another can give 
rise to different understandings. Once detached from 
the framework of questions that gave rise to their 
existence, data easily become forces of dissolution — 
the sound and the fiiry, signifying everything and 
nothing. 

T hen there are the data not collected — the lacunae in 
the spectrum of information that would yield a 
better understanding, for example, of what students 
learn, how faculty spend their time, whether the insti- 
tution’s investment in such things as graduate education 
or intercollegiate athletics bestows financial or other 
benefits on the institution as a whole. These are blind 
spots in the vision of the future, the questions left 
unasked about an institution’s capacity to pursue 
newly defined needs or opportunities. In academic 
settings data are not collected for two reasons: “We 
really don’t want to know because it will make us 
change our minds,” or “We don’t want someone else to 
know for fear the result would further erode the acad- 
emy’s independence and autonomy.” Most faculty 
members do not want to be measured, believing that 
scholarly endeavor is not capable of a quantitative 
rendering. They don’t want their departments com- 
pared with others either within or outside their institu- 
tions, fearing that faulty judgment will lead to fewer 
resources. Most presidents and their provosts are 
inherently suspicious of external calibrations, having 
learned long ago that purported measures of efficiency 
and productivity too often result in formula allocations 
that limit their institution’s prospects. In both cases, 
the best defense is not to collect data that, in the hands 
of others, must inevitably lead to invidious comparison. 

M ost data generated by a college or university 
attest to where the institution has been. What is 
too often missing is a data-driven framework that 
would allow the institution to ask: “Where are we 
likely to be tomorrow or next year or ten years from 
now?” Very few institutions, for example, can say 
with much certainty how much of their student market 
is placed at risk by the presence of new, often for- 
profit providers of postsecondary education. Even if it 
has not encountered significant competition from 
alternative providers, an institution must periodically 
ask key questions about its market and its competi- 
tion: “How big is that market? Who else serves it 
now? What is our market share, and how could that 



change in the future? How big of a margin do those 
other providers realize?” Knowing the answers to 
these questions is a critical step in determining the 
prospects for success in a new market as well as the 
likelihood of sustained competitiveness in an old one. 

In many colleges and universities the inability to 
use data strategically stems from a failure to understand 
how much their continued vitality depends on their 
bottom lines. All too often, higher education institutions 
seem willing to accept on faith that a decision is the 
right one, simply because it is consistent with their 
mission and has been reached through collegial dis- 
cussion. Many institutions today are facing financial 
crises stemming from earlier decisions made with lit- 
tle or no consultation of strategic data. The financial 
losses incurred from the clinical activities or hospitals 



Many institutions have a fundamental 
aversion to the language and concep- 
tual framework of business enterprise, 
and they are loath to organize their 
data systems in terms of customers, 
markets, product lines, or activity- 
based costing. 



of some universities with medical operations demon- 
strate the consequences of judgments made with too lit- 
tle consideration of markets and economic impact. As 
other institutions extend their product lines into activ- 
ities like distance learning, applied research, non- 
credit certification, and industry training, the ability to 
keep track of both traditional and non-traditional 
activities will become even more important to institu- 
tional leaders. 

L ike any business, higher education institutions 
need to know how much time and effort are going 
into the development of these new activities, how 
much cross-subsidy is taking place, and the extent to 
which new initiatives strain the basic capacity and 
efficiency of different units. But many institutions 
have a fundamental aversion to the language and con- 
ceptual framework of business enterprise, and they 
are loath to organize their data systems in terms of 
customers, markets, product lines, return on invest- 
ment, depreciation of products and knowledge, or 
activity-based costing. 
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Operational Data 

If strategy entails the ability to foresee and 
respond to new opportunity, part of that skill derives 
from a basic reading of vital signs in one’s own orga- 
nization. The good news is that colleges and universi- 
ties have gotten better at developing and using opera- 
tional data, largely because they have had no other 
choice. The cost of making mistakes has become 
more telling as the public increasingly holds higher 
education to the same performance standards applied to 
other industries providing services for a fee. 
Institutional managers can no longer afford to be 
vague about budgetary expenditures, numbers of full- 
and part-time students, numbers of personnel, or the 
numbers on student applications, admissions, and 
matriculations. Almost everywhere institutional 
research functions have been beefed up. The data sys- 
tems monitoring operations are now responsive to 
most administrative demands for timely and accurate 
information, giving, as one administrator put it, 
“ready access to what is there.” 

Academic managers have also become more 
adept at reading and interpreting operational data, 
learning to recognize the tell-tales of impending prob- 
lems: slow processing times during registration; 
decreased inquiries as a harbinger of an enrollment 
shortfall; an increase in the number of students who 
have deferred degree requirements. Good academic 



Nothing can be less productive than 
having alarms sounded in the media and 
other public venues on the basis of 
numbers intended as early-warning 
beacons for institutional leaders. 



managers have also learned to trust systems and sub- 
ordinates, relying on them to gather the necessary 
indicators rather than insisting on getting all the raw 
data “first-hand.” 

W hat most skilled academic officers want first 
are unambiguous indicators — warning lights 
on what many have come to conceive as a leadership 
dashboard. Some of these fall under the heading of 
academic and scholarly capacity: such things as 
unplanned change in student enrollment or retention; 
increased difficulty in recruiting or retaining faculty; a 
downward shift in the number of research proposals 



submitted to external funders; decreased competitive 
scholarly performance; loss of community-based 
“preferred partner” relationships. Other indicators are 
financial: shortfalls in reaching fundraising targets; a 
decrease in generated research funding; increased 
incidence of departments over-spending their bud- 
gets. A third set of warning beacons might be termed 
external: changes in state activity resulting from shifts 
in public opinion or political action; changes in media 
coverage; or a change in the placement rate of gradu- 
ates from undergraduate or graduate programs. 

Most of the time the information necessary for an 
early-warning system is readily available, and leaders 
with access to such information are expected to act — 
to know what the problem is and how the institution 
ought to respond. Institutions in this early-warning 
frame of mind can be expected to develop a culture of 
watchful competitiveness among units — a sensitivity 
both to general danger and to particular weakness that 
might be turned to another unit’s advantage. Given 
this awareness, most provosts and deans quickly learn 
not to sound public alarms too early or appear to be dis- 
aster mongers. The question most seasoned academic 
leaders have to answer sooner or later is: How big is the 
back seat? How public should the warning signs be? 
Who should have direct access to the information the 
indicators provide? 

O n one side is the academic tradition of open 
access and data transparency. In a culture of 
shared governance, it is hard to deny faculty access to 
the same indicators the administration uses to identify 
potential problems. The other side is populated by 
realists who recognize that in complex institutions 
and competitive cultures such indicators can become 
instruments of attack as well as warning. Few indeed 
are the academic administrators who have not made 
summary data available to a key committee or the fac- 
ulty as a whole, only to find the indicators, regardless 
of the stated purpose of their release, suddenly mar- 
shaled in a broad, often mean-spirited attack on the 
administration itself. What gets lost in these moments 
of combat are the underlying assumptions that give 
the indicators meaning as well as the caveats that 
inevitably accompany the release of technical data. 

Beyond the impact that a warning sign may have 
among an institution’s departments and units is the 
effect of such information on external constituencies. 
Nothing can be less productive than having alarms 
sounded in the media and other public venues on the 
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basis of numbers intended as early-warning beacons for 
institutional leaders. The leader’s challenge is to create 
a space for discussing warning data in a productive 
way, without creating an “I-win-you-lose” environ- 
ment among the parties involved. In practical terms, 
this means that leaders must reserve control of early- 
warning data. 

Indicators are predictors of likely performance if no 
corrective action is taken — provided, of course, the 
indicators are well defined. Ordinarily, a warning indi- 
cator is a signal for a quiet review of the potential 
problem with the leadership of a given unit rather than 
a cause for public alarm, much less for emergency 
measures. In those times when the early-warning data do 
precede an unfavorable change in circumstance, the 
question is how to convey this information responsibly 
to both internal and external communities. What data 
should become public, when? Who makes them public? 
An academic leader should have sufficient control 
over key performance data to be able to determine the 
answers to such questions along with the parties 
directly involved, rather than being co-opted by an 
information network that keeps nothing under wraps. In 
higher education, as in other strategic operations, not 
everyone can know everything all the time. 



In Search of a Culture of Data 

The fact that the release of data so often acceler- 
ates and even exaggerates political conflict suggests 
that most institutions lack a genuine culture of data. At 
most institutions, comparatively few members of the 
campus community have the skills or perhaps just the 
patience to interpret the management data presented to 
them. Those academic leaders who succeed in making 
their campuses data smart often do so by providing 
cues to responsible interpretation before presenting 
data for public review and comment. Having taken the 
precaution of presenting data in a way that fosters 
understanding without sowing confusion, what is also 
required is a sense of the story the data tell — the nar- 
rative that provides context as well as meaning. It is a 
story that links the past to the present while sounding 
themes for the future as well. Often this step consists 
simply of recounting the institutional purpose and 
goals that have given rise to the data presented. In 
saying out loud why the data were assembled, how 
they fit into the sustaining goals of the institution, and 
what they say about the progress in reaching those 



goals, the knowledgeable academic leader — presi- 
dent, provost, dean, department, committee or task 
force chair — draws on his or her own sense of the data 
as well as past efforts to establish shared expectations 
among colleagues as to how data should be understood. 

J ust as persistent is the problem of building consistent 
crosswalks between separate databases. Even when 
institutions understand the problem, they find it diffi- 
cult to muster the political will to resolve or override the 
territorial disputes over who owns which data. When 
installing new computer systems, the impulse is still too 



Those academic leaders who succeed in 
making their campuses data smart often 
do so by providing cues to responsible 
interpretation before presenting data 
for pubhc review and comment. 



often to customize so that the new reports and 
processes “look like” the old ones, signifying that the 
institution remains sensitive to those local customs, 
traditions, and definitions that in the past have governed 
the collection and use of data. 

Not surprisingly, then, better transactional data 
have not yielded less costly institutions. Today, uni- 
versities and colleges expend more time, effort, and 
money than ever before in gathering data. 
Investments in technology have made it possible for 
institutions to reduce the amount of time spent assem- 
bling numbers on basic transactions. For all that, 
higher education institutions still have not learned to 
organize and use data effectively for internal deci- 
sions or public accountability. 



A Matter of Strategy 

In the end, however, it is not efficiency per se but 
effective strategies that matter in enterprises that are 
expected to be both mission centered and market 
smart. While transactional data can be expected to 
supply the basic building blocks of an information 
system designed to aid strategic decisions, what the 
academic community requires most of the time is a 
clear sense of the direction the institution is heading. 
Here the problem, ironically, is the inability of most col- 
leges and universities to look over the horizon, several 
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years out, and imagine the opportunities for new and 
different roles they might play in fulfillment of their 
missions. Strategically defined data ought to help 
institutions think outside the box of their current cir- 
cumstances. What those responsible for steering a 
strategic course — chairs, deans, provosts, and presi- 
dents — require, simply put, are indicators that help 
them anticipate opportunities as well as problems. 

The strategic issues that concern most academic 
leaders fall under comparatively few management 
domains: 

1. Units: the financial and academic well-being of 
individual schools, departments, institutes, and 
programs. 

2. Research/Product Lines: the vitality and perfor- 
mance of the research enterprise and of other 
programs, . services, or products in which the 
institution has invested. 

3. Markets: what new competition has entered 
existing markets for students or sponsored 
research, and what new market opportunities 
have appeared. 

4. Learning: the performance of academic pro- 
grams; what graduates learn, and how they use 
what they have learned. 

5. Talent: the richness of the pool from which the 
institution draws its current and future faculty, 
and the institution’s capacity to support faculty 
development. 



The kinds of capacity questions an academic 
leader would want answered about these five domains 
are also comparatively few: 

A. Economics: the extent to which work in this 
domain is well defined, services are delivered 
efficiently, costs are contained, return on invest- 
ment is realized, etc. 

B. Competitive Capacity: the degree to which the 
institution’s strength in this domain helps it 
compete with others in its market. 

C. Collaborative Capacity: the extent to which the 
different units within this domain work together 
to leverage their strengths. 

D. Deployment: the extent to which human and 
financial resources are aligned within this 
domain to fulfill both unit and institutional 
goals. 

E. Mission Centeredness: the extent to which suc- 
cessful performance within this domain fulfills 
the institution’s particular mission and adds 
value through its contribution to the public 
well-being. 

This arrangement of domains and capacities 
yields a matrix that can be used to support the making 
of strategic decisions. Casting the numbered domains 
in rows and lettered capacities in columns produces the 
following conceptual scheme: 
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Viewed broadly, it is a matrix for charting a strate- 
gic course. Taken literally, the matrix provides a simple 
gauge of the availability and quality of data to support 
decision making. Each cell of the matrix invites the 
provost, dean, or department chair to ask: “How exten- 
sive — and how reliable — are the data available to 
gauge the subject that a given cell defines?” How solid 
and broad is the base of information to determine, for 
example, the collaborative capacity of a set of depart- 
ments, the mission centeredness or competitive capac- 
ity of an institution’s learning programs? 

Ultimately the more important element of the 
matrix derives from its potential first to identify and then 
to sharpen an institution’s understanding of the oppor- 
tunities and challenges it faces. Aligning domains and 



Colleges and universities have defied 
every attempt to develop industry-wide 
standards, in part because each institu- 
tion wants its own definitions, its own 
measures of quahty and performance, 
and hence its own data systems to 
prevail. 



capacities in this way can yield a more precise sense of 
the conversations in which a leader might want to 
engage concerning the state of the institution — past, 
present, and future. As heuristic, this exercise can gen- 
erate insights leading to a broader understanding of 
data as part of a continuing narrative of institutional 
progress toward longer-range goals. “The matrix,” as 
one administrator has observed, “serves me primarily as 
a generator of questions. It helps to frame the conver- 
sations I would like to foster within the institution as we 
consider our strategic options. It helps me to identify the 
pieces of our institutional story that I need to know 
more about.” In short, it is a conceptual tool that can link 
data more closely to a shared frame of meaning. 



Gauging Collective Performance 

One of the things that distinguishes higher educa- 
tion from virtually every other industry is its passion for 
customization. Colleges and universities have defied 
every attempt to develop industry-wide standards, in 
part because each institution wants its own defini- 



tions, its own measures of quality and performance, and 
hence its own data systems to prevail. 

One result of this independent-mindedness is that 
higher education as a whole cannot answer the basic 
questions that have come to occupy national attention. 
Because institutions gather data primarily in the service 
of their own particular needs, the industry as a whole 
can say very little in response to questions about its 
overall success and capacity. There are no data that 
track such things as student persistence or completion 
from a system-wide perspective — no protocols to 
account for the growing phenomenon of students who 
stop in and out of higher education, enrolling intermit- 
tently in different programs at different institutions, in 
pursuit of knowledge and skills for purposes that may 
change at different times of their lives. The industry as 
a whole has difficulty bringing evidence to bear on 
questions about the success of its learning programs or 
the value that a college education contributes to the 
lives of its students; and it cannot say with certainty 
how well it is meeting public expectation in terms of 
access, participation, or affordability. 

W hat is required is a database that charts the perfor- 
mance of the system as a whole in meeting the 
nation’s expectations for higher education. While the 
need for such information is now broadly affirmed, the 
means and the details still await resolution. What troubles 
most institutional leaders is the notion of an imposed 
data collection mechanism — something linked to the 
Institutional Postsecondary Education Data System 
(IPEDS) or another federal initiative — that would mea- 
sure the overall capacity of the system by summing up the 
capacities of each and every institution. 

We think these institutional leaders are right to 
worry. Most proposals for such a collection instrument 
have turned out to be expensive as well as intrusive, 
leading ultimately to the kind of scorekeeping in 
which the winners are few and the losers many. 

The answer instead is to separate the questions of 
institutional performance from measures of system 
capacity. The former is a matter of providing good 
consumer information — something the market is 
already insisting upon. Our bet is that much sooner 
than most institutions imagine there will be a variety of 
competing ratings, each providing what might best be 
called a consumer report for higher education. 

The question of system capacity, however, will 
not be answered by the market. What is required here 
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is a broadly defined, regularly maintained national 
database that focuses on students/graduates rather 
than institutions, and that measures the key public 
policy issues of access, performance, and results. 
Who is and is not being shut out of higher education? 



Higher education ought to take upon 
itself the funding and maintaining of a 
mechanism for collecting paneled data 
on how well the system is serving the 
education needs of the nation as a whole. 



Who starts but does not finish, and why? What is 
being learned, and for what purpose? How well are 
graduates being prepared for the world of work? Are 
they simultaneously learning the responsibilities of 
citizenship? Are they acquiring the habits of mind that 
promote lifelong learning? 

Ironically, data for answering these questions 
have been sporadically available from the federal gov- 
ernment. In 1980, the National Center for Education 
Statistics (NCES) began tracking a representative sam- 
ple of high school sophomores for High School and 
Beyond^ interviewing them as seniors in 1982 and 
every two and then four years through 1 992. Alas, that 
sample was then abandoned in favor of a second sam- 
ple that began tracking a set of eighth graders starting 
in 1988 for the National Longitudinal Survey (NLS). 
Now it is unclear whether this sample will continue to 
be tracked, as the interests of NCES continue to shift in 
response to changing political circumstances. 

H igh School and Beyond and NLS demonstrate 
that it is possible to use data derived from this 
tracking of a cohort of individuals to produce a valid 
snapshot of how well the nation’s system of higher 
education is serving its student-customers. The samples 
were large enough to allow researchers to focus on 
different market segments — for example, community 
colleges, liberal arts colleges, research universities, or 
even for-profit institutions that primarily serve adult 
populations. The cost of these efforts was substantial, 
but not outrageous. Had they been continued and then 
added to with the forming of additional cohort panels 
(probably every five years), the result could have been 
a sustained capacity to ask the key questions the pub- 
lic and their elected officials have increasingly asked of 
the nation’s investments in higher education. 



H ere we have a simple proposal to make. Rather 
than depending on the largesse and whims of the 
federal government, higher education ought to take 
upon itself the funding and maintaining of a mechanism 
for collecting paneled data on how well the system is 
serving the education needs of the nation as a whole. A 
simple back-of-the-envelope calculation suggests that 
if all accredited institutions were to pony up an average 
of $500 per institution per year, sufficient funds 
would be available. A truly independent agency 
would have to be created to ensure the reliability and 
consistency of the data instruments and the integrity of 
the data. As with High School and Beyond^ some of the 
cohorts would have to be identified while still in high 
school. The data themselves would have to be made 
broadly available to the research community; local, 
state, and federal agencies charged with overseeing 
the nation’s investments in higher education; the 
media; and indeed anyone with an interest in the 
subject, including the colleges and universities 
themselves. 

This effort would provide a set of indicators shed- 
ding light on the success of postsecondary learning 
and the value that a completed degree or coursework in 
higher education adds to the life of students. Such a 
database would also make possible some degree of 
experimentation with samples of students to help 
understand more clearly the impact of different variables 
on educational attainment and subsequent achieve- 
ment. What would be the effect, for example, of pro- 
viding a slightly greater amount of student financial 
aid to a given subset of students than they might ordi- 
narily expect to receive? Would this additional aid 
translate into more effective learning outcomes for stu- 
dents in terms of educational achievement and later 
experience? A national longitudinal database tracking 
cohorts of students could show more clearly how dif- 
ferent kinds of financial aid awards affect learning and 
achievement, in higher education and beyond. The 
findings from such experiments would help both policy 
officials and individual institutions to develop more 
targeted strategies for the allocation of financial aid to 
yield greater educational achievement and success in 
life. Other experiments, concerned more with peda- 
gogy and learning than with how students pay for and 
benefit from higher education, would also be possible. 

Some might contend that a more useful way of 
measuring the performance of higher education is to 
assemble data in terms of institutional type. The natural 
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tendency of universities and colleges is to calibrate 
their own performance in the context of either peer 
institutions or of “aspirational” institutions that offer an 
image of what they themselves would like to become. 

T he trouble with such benchmarking is that it too 
often reinforces the habit of understanding higher 
education’s performance only in terms of particular 
segments rather than from the standpoint of the enter- 
prise as a whole. For any given institution, confining 
attention to a particular peer or aspiration group has the 
effect of narrowing the field of vision and divesting 
oneself of responsibility for higher education’s perfor- 
mance in the broader context of issues important to 
the enterprise’s external stakeholders. Indeed, it is 
often the preoccupation with peer and aspiration 



The national database we envision would 
not be for the purpose of comparing 
individual institutions. Its focus would be 
on students and the impact that their 
enroUment in higher education has on 
their learning and later experiences. 



groups that drives the growth of costs in institutions, as 
each scrambles to make sure it keeps pace with the 
others in salary and other measures. Higher education 
institutions of all kinds need to look beyond their own 
back yards to understand the larger domain of goals and 
issues confronting the industry as a whole. 

To repeat, the national database we envision 
would not be for the purpose of comparing individual 
institutions. Its focus would be on students and the 
impact that their enrollment in higher education has on 
their learning and later experiences. 

W hat would any individual institution hope to 
gain from this yearly expenditure of funds? 
First of all, a national project to report on the industry’s 
performance would send a strong signal to policy- 
makers and other external stakeholders that higher 
education intends to be responsive to their concerns. If 
higher education can demonstrate the ways in which, as 
an industry, it is doing a good job, every institution will 
reap the benefit of increased public confidence. 

Second, a project of this sort would help to 
strengthen the consciousness among all institutions of 
the broader public purposes their individual missions 



help to fulfill. The ultimate benefit might well be a 
greater willingness to work together, both within and 
across different types of higher education institutions, 
to build strategies that yield improvement in measures 
such as access, affordability, persistence, and effec- 
tive learning. 

F inally, though the database would not yield institu- 
tional comparisons, it would produce benchmarks 
against which an institution could measure its own 
learning capacities. Without fear of invidious compari- 
son, a college or university could administer parallel 
instruments to samples of its own graduates, confident 
that the interpretation of those results would take place 
within an appropriately comparative framework. 

An accounting of this sort captures only a part of 
the contribution that higher education institutions 
make to society. A more complete description of 
higher education’s capacity and its effectiveness in 
fulfilling the public good would also include its pro- 
grams of research, outreach, and other services. We 
believe, however, that a database focusing on how 
well students are being served by the nation’s higher 
education institutions would be a promising start; as 
such, it is a collective effort in accountability that 
warrants the participation of every university and col- 
lege in the nation. 



Making Data Do It 

Today’s higher education leaders are beneficia- 
ries of the fact that the industry has come to have bet- 
ter control and understanding of the basic transac- 
tional data that track its operational procedures. 
Virtually every institution has sharpened its attentive- 
ness to these measures, largely in response to 
increased scrutiny and demands for accountability 
from without. Those responsible for the academic 
management of institutions are also finding that their 
strategic dashboards have increased in scope as well as 
in quality and reliability. Yet higher education contin- 
ues to lack a culture of data in the fullest sense, in part 
because it has yet to resolve the ambivalence about 
what data should tell and how they should be used in 
institutional decision making. 

Building a culture of data will require that all 
institutions become more willing to adopt shared con- 
ventions in data collection and analysis, even if that 
means initiating practices that differ from the past. It 
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will require that members of academic communities 
distinguish more clearly between the repeated gather- 
ing and testing of data which underlie scientific dis- 
covery, on the one hand, and the use of data for mak- 
ing institutional decisions on the other. Some lessons 
from our own experience would suggest the following 
as principles worth the consideration of every academic 
institution. 

M ake data the instruments of prospective strategy, 
not weapons of turf defense and opposition. 
The very principles that define data of the latter sort 
often constitute non-answers to the central strategic 



Ultimately, an academic community 
that learns to use data responsibly must 
move beyond a conception of strategy 
as the aversion of threats and danger to 
an understanding of strategy as the 
anticipation of change and the building 
of institutional capacity to pursue new 
opportunity. 

questions facing an institution, its schools and depart- 
ments. Every institution could benefit from spending 
less time gathering data for proprietary purposes, and 
more time identifying and analyzing data that gen- 
uinely speak to questions of institutional strategy. 
Now is the time for campus communities to reach 
beyond the mindset that allows any call for “more 
data and analysis” to cripple an institution’s ability to 
make and act on strategic decisions. At the same time, 
institutions must guard against making impulsive 
commitments of resources to a project without ever 
considering key indicators of its advisability. 

Academic leaders ought to he those who demon- 
strate by example what it means to make effective 
use of data. Presidents, provosts, deans, department 
chairs, and senate chairs are in a natural position to 
show what it means to make judicious use of informa- 
tion in pursuit of a well-conceived strategic course. 
They can convey to an academic community a 
responsible sense of what data do and do not tell; they 
can show when to draw upon quantitative measures, 
and when to move beyond these indicators to make 
decisions on the basis of more intuitive responses to a 
range of information. A leader who holds all decisions 



hostage to continual demands for more data will very 
likely feed a culture of defensiveness in an institution, 
causing every unit to amass data like so many ramparts 
against an incursion. Making responsible use of data 
means knowing when it is time to act, even though it 
might be possible to attain more information. It means 
not sounding all alarms on the basis of a single early 
warning from a dashboard indicator. On the other 
hand, it means not waiting until seawater is rushing into 
the captain’s cabin before ordering all hands on deck. 

Academic leaders should act on the recognition 
that data take on meaning only as they become parts 
of a continuing story of an institution goals and its 
progress in reaching them. Just as no cartographer 
would produce a map that did not include coordinates 
or a key to identifying its conventions of representation, 
any academic leader who presents data to an institu- 
tional community should take steps to ensure that 
those data can be clearly understood in the context of 
a continuing narrative of institutional transformation. 
By a variety of means — including the simple strategy 
matrix we have presented — academic leaders can help 
their communities understand data as a way of telling 
and extending an institution’s “story” from the past 
through the present to its future. Ultimately, an acade- 
mic community that learns to use data responsibly 
must move beyond a conception of strategy as the 
aversion of threats and danger to an understanding of 
strategy as the anticipation of change and the building 
of institutional capacity to pursue new opportunity. 

F inally, higher education institutions, both indi- 
vidually and collectively, need to work more 
deliberately to develop a national database in service 
to their external constituencies. The concept of cus- 
tomer service has come to bloom more slowly in 
higher education than in other industries. While most 
individual colleges and universities have developed 
data systems that monitor their performance in meeting 
the needs of their own student-customers, the industry 
as a whole continues to lack a means of reporting its 
collective performance in fulfilling the nation’s 
expectations for higher education. By taking the lead in 
building a national database centered on its own col- 
lective performance, colleges and universities would 
take a giant step toward viewing service to the public 
as truly central to their missions — and would thereby 
demonstrate higher education’s collective commit- 
ment to take responsibility for its own actions. 
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